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Abstract —The two-receiver broadcast packet erasure channel 
with feedback and memory is studied. Memory is modeled using 
a finite-state Markov chain representing a channel state. The 
channel state is unknown at the transmitter, but observations 
of this hidden Markov chain are available at the transmitter 
through feedback. Matching outer and inner bounds are derived 
and the capacity region is determined. The capacity region 
does not have a single-letter characterization and is, in this 
sense, uncomputable. Approximations of the capacity region 
are provided and two optimal coding algorithms are outlined. 
The first algorithm is a probabilistic coding scheme that bases 
its decisions on the past L feedback sequences. Its achievable 
rate-region approaches the capacity region exponentially fast in 
L. The second algorithm is a backpressure-like algorithm that 
performs optimally in the long run. 

I. Introduction 

In many communication protocols, information transmission 
is done in a packet by packet manner, and the receiving devices 
either correctly receive the transmitted packet or detect an 
erasure. Packet erasure channels (PECs) model such systems. 
Broadcast PECs (BPECs), in particular, are interesting models 
to study the broadcast nature of wireless systems. 

The capacity region of the general broadcast channel (BC) 
remains unresolved both without and with feedback. It was 
shown in ID that feedback does not increase the capacity of 
physically degraded BCs. It is known that feedback increases 
the capacity of general BCs and even partial feedback can help 
12 - 0 . 

Remarkably, capacities of memoryless BPECs are known 
both with and without feedback for two receivers. The former 
is a special case of the work in 0 which characterized 
the capacity of degraded broadcast channels. The latter was 
derived in J7J. Generalizations of the idea presented in 
0 have led to optimal coding schemes for BPECs with 
three receivers and near-optimal coding schemes for more 
receivers in [ 8l— llTOI . The schemes are feedback-based coding 
algorithms that are based on network coding techniques and 
have also influenced related recent works on continuous output 
broadcast channels with feedback and fading fTTI . fl2ll . 

This work studies BPECs with feedback and channel 
memory. The problem is motivated by the bursty nature of 
erasures in practical communication systems, e.g. satellite 
links na-na. When there is no feedback, one can 
use erasure correcting codes for memoryless channels in 
combination with interleavers to decorrelate the erasures. 
Interestingly, feedback enables more sophisticated coding 


techniques fl6l . In a related work, ifTTI derived an 
infinite-letter capacity characterization for two special cases 
of the general BC with feedback, memory, and unidirectional 
receiver cooperation. 

We model the memory of a channel by a finite state machine 
and a set of state-dependent erasure probabilities. This is a 
well-studied approach for wireless channels (see fi~8l Chapter 
4-6], lfl9l and the references therein). 

In a previous work f20l , we studied BPECs with feedback 
and memory under the assumption that the channel state is 
causally known at the transmitter. We derived close inner 
and outer bounds on the capacity region. This problem was 
also studied in parallel by Kuo and Wang in ED- ED- 
They proposed a new coding strategy and characterized its 
corresponding rate region. The outer bound in (20]] and the 
inner bound in ED, ED match and thus characterize the 
capacity region. This paper extends the previous results to 
the case where the channel state is no longer observable at 
the transmitter (except through the receivers’ feedback) and 
thus evolves according to a hidden Markov model from the 
transmitter’s point of view. 

Our contribution in this paper is as follows. We propose 
an outer bound on the capacity region that has an n-letter 
characterization in terms of a feasibility problem. This region 
is not computable because its computation needs an infinitely 
large n, and the number of parameters in the corresponding 
feasibility problem grows exponentially in n. Nevertheless, 
under mild conditions, we find a sequence of approximations 
on this region, Cjg em (L), for every L > 1. The L th order 
approximation of the region approaches the outer bound 
exponentially fast in L. For every L, we propose a probabilistic 
encoding strategy at the transmitter that bases its decisions 
on the past L feedback symbols and achieves the L th order 
approximation of the outer bound. Finally, a deterministic 
algorithm is outlined that bases its decisions on the entire past 
feedback symbols, is implementable, and optimal in the long 
run. 

This paper is organized as follows. We introduce the system 
model in Sec. UD An outer bound on achievable rates is derived 
in Sec. HI Achievable schemes are discussed in Sec.lIVIbefore 
we conclude in Sec. m Proofs and detailed derivations can be 
found in the Appendix. 
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Fig. 1. Block diagram for the BPEC. The box marked with d = 1 represents 
a delay of one time unit. 


S n is stationary. The state-dependent erasure probabilities are 
specified through Pz t \S t that does not depend on /:. We permit 
arbitrary correlation between Zi it and Z 2t . The erasures 
Z" are correlated in time in general, hence the channel has 
memory. The sequence Z” is stationary due to stationarity of 
S n . 

After each transmission, an ACK or negative ACK (NACK) 
feedback is available at the encoder from both receivers. As 
opposed to our previous work in | 20 l , we assume no separate 
feedback of the channel state; i.e., the channel state is not 
known at the transmitter. The channel input at time t may be 
written as 


II. Notation and System Model 

A. Notation 

Random variables (RVs) are denoted by capital letters. A 
finite sequence (or string) of RVs X\,X\, ..., X n is denoted 
by X n . Sequences can have subscripts, e.g. X''- is shorthand 
for X^i, JVj-, 2 , • ■ • • Xj, n . Vectors are written with underlined 
letters, e.g., Z t = (Zi^, Z 2 ,t). Sets are denoted by calligraphic 
letters, e.g., X. The indicator function !{■} takes on the value 
1 if the event inside the brackets is true and 0 otherwise. 
Conditional probabilities are written equivalently as Pr[X = 
x\Y = y ], P x \y{x\v ) or sometimes P(x\y) if the involved 
RVs are clear from the context. The conditional expectation 
of a function / of a random variable X given another random 
variable Z is itself a random variable and is written as 
E[f(X)\Z], Using the law of total expectation E[f(X)\Z] = 
E [E[f(X)\YZ] | Z]. Note that if X-Y-Z forms a Markov 
chain, we can write E[/(X)|Z] = E[E[/(X)|V] |Z]. 

B. System Model 

A transmitter wishes to communicate two independent 
messages W\ and W 2 (of nR\, nR 2 packets, respectively) 
to two receivers Rxi and RX 2 over n channel uses. 
Communication takes place over a BPEC with memory and 
feedback as shown in Fig. [T] and described below. 

The input to the BPEC at time t is denoted by Xt € X, 
t = 1 The channel inputs are packets of £ bits; i.e. 

X = F q with q = 2 l , and £ 1. All rates are measured in 

packets per slot and entropies and mutual information terms 
are with respect to logarithms to the base q. 

The channel output at Rx ; at time t is written as Y. j t £ V. 
j £ {1,2}, where y = X U {E}. Each Yj t is either X t 
(received perfectly) or E (erased). 

We define binary RVs Z. jt , j £ {1,2}, t = 1 ,...,n, to 
indicate erasure at Rxj at time i; i.e. Z J t = l{k},t = E}. 
Yjt can be expressed as a function of X t and Z ]t but Y ) t 
also determines Zj t . We collect {Z[_ t . Z 2 t) into the vector 
Z t , with Z t £ Z, Z = {(0,0), (0,1), (1,0), (1,1)}. 

The memory of the BPEC under study is modelled with a 
finite state machine with state St at time t. The underlying 
homogeneous finite state Markov chain is assumed to be 
irreducible and aperiodic with state space S and steady-state 
distribution 7r s , s £ S. The initial state So is distributed 
according to the steady-state distribution, so the sequence 


X t = f t (W 1 ,W 2 ,Z t ~ 1 ). (1) 

The probability of erasure in each channel state is 
described based on the past channel state. Nevertheless, at 
the transmitter, the channel state is not available and thus the 
probabilities of erasure events depend on all the past observed 
feedback messages, z t_1 , as follows: 

£ 12 ( 1 * X ) = Pz_ t \zy~ 1 i 1 , 1 |“* 1 ) 
ei2(z t - 1 ) = Pz t \zt-i(0,l\z t - 1 ) 

£i 2 (l* _1 ) = Pz t | \z *~ 1 (1,0 |l t_1 ) 

£i(l t - 1 ) = £i2(l t - 1 ) + e 1 2(f- 1 ) 

e 2 (z t - 1 ) = e 12 iz t - 1 ) + e l2 (z t - 1 ) ( 2 ) 

Each decoder Rx ; has to reliably estimate W :j from its 
received sequence YJ 1 . A rate-pair ( Ri,R 2 ) is said to be 
achievable if the error probability Pr[W! 7 ^ W\,W 2 7 ! W 2 ] 
can be made to approach zero as n gets large. The capacity 
region C™ m is the closure of the achievable rate pairs. 

III. The Outer Bound 

Define C™™, for every integer n, as the closure of 
all rate pairs ( R\,R 2 ) for which there exist variables 
2/fe* -1 ), l* -1 £ Z t_1 , t = 1 ,...,n, such that 

0 < x(z t_1 ), r/(z* _1 )< 1, Vf = 1,V z_ t_1 G Z t_1 (3) 

1 n 

Ri<~Y^ ^2 ■Pz*-ife t_ 1 )(l-£ife* _ 1 ))a;(l t_1 ) (4) 

' zt-'GZ *- 1 

1 n 

Ri<-J2 E Pz^-^-e^Ml-y^- 1 )) (5) 

n— 

t —1 z^GZ *- 1 

I ” 

f? 2 <~E E • p z t - 1 fe‘ _ 1 )( 1 _e 2 fe* _ 1 )) 2 /fe t_1 ) ( 6 ) 

1 ” 

1 ? 2 <-E E 

n z —' z —' 

Define C™ m as the limsup of C™™, when n —> 00 . With 
similar steps as in lf20l Section IV], we have: 

Theorem 1. Any achievable rate pair ( R\,R 2 ) is such that 
(Ri - 6, R 2 - 5) £ C£ em , for 6 > 0 . 

The proof can be found in Appendix [A] 
























Remark 1. To compare this result with the outer bound in 
f20l . where the channel state was strictly causally known at 
the transmitter, we note that Theorem Q] is a very general form 
of l20l with infinite states. Each state represents the state of 
the system at time t, with a feedback Z_ t_1 received. 

Remark 2. Consider a Gilbert-Elliot model with state space 
S = {GG, GB, BG, BB}, where G and B refer to a good and 
a bad channel state at each user, respectively. Consider the 
special case where we always have an erasure in state B and 
no erasure in state G. Even when there is no separate channel 
state feedback, this channel model may be considered as a 
special case of l20l , for the transmitter has full knowledge 
about the past channel state upon receiving the feedback. It is 
not hard to show that, in this special case. Theorem [T| reduces 
to the outer bound in l20l . 

Theorem Q] establishes an outer bound on the capacity 
region. The above characterization is, however, uncomputable. 
We find an approximation of the region next. 


A. Approximation of the Outer Bound 

The derived outer bound has an infinite-letter 
characterization because S t -i is not fed back to the 
transmitter, but an estimate of it is available through the 
feedback symbols Zf _1 . From the transmitter’s perspective, 
the predicted erasure probabilities Pzjz*- 1 depend on all 
the past Zf- 1 . Intuitively, one expects that the effect of 
past feedback diminishes rapidly. We thus approximate the 
infinite-letter bound in (j3j - ([7]) with a finite-letter bound 
where channel erasure events depend only on the past finite 
L feedback symbols. The predicted erasure probabilities for 
time slot t are as follows: 


p Utl~‘ x ) = ^Pz t \sMt\s)P St \z^{s\± *) (8) 

sG«S 

Pz t | s t does not depend on t, but Pspz*- 1 does. In practice, 
one observes that the latter distribution is “close” to P Q v >- ' 
which predicts channel states from the past L feedback 
symbols if L is reasonably large. This is made precise by 
the following theorem which is adapted from l23l Theorem 
2 . 1 ]. 


Theorem 2 (cf. 1231 Theorem 2.1]). Suppose that all entries 
of both the state transition matrix Ps t \s t -i ar 'd the distribution 
matrix Pz t \s t are strictly positive. For any observed sequence 
z t_1 £ Z t_1 , we have 


E | p s t \zt-^{s\^ *) - Pg t \zfff 

sG«S 


mzb <2(1 


r) L , 


where <j>0 depends on the smallest entry of matrix Ps t \s t ~i 
and the ratio of the largest and smallest values in matrix Pz t \s t - 
Z?t-i is shorthand for Z t _ L ,Z t _ L+1 ,.. .,Z_t-i- 

Remark 3. The theorem in its original form [23] Theorem 
2.1] is more general and only requires the r-th order transition 
matrix Ps t \s t -r to t> e strictly positive. We use the weaker form 
here for simplicity. 


The following corollary, derived in detail in Appendix [B] is 
the basis of the approximated outer bound: 

Corollary 1. Under the conditions of Theorem [2] the total 
variation between and P z \ Z '—' ' s bounded for any 

sequence £ Z t_1 by 

E l p (^‘ -1 ) - p iSt\d-\)\ < 2(1 - ^) L - (9) 


Hence an approximate characterization of the outer bound 
C™ m is given by the following feasibility problem. 


0 

< 

x(z L ), 

,yU L ) < 1 , 

V 

z L £Z 

L 

(10) 

Ri 
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E 

p za (± L 
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- ei (z 

L )Mz l 

0 + c L 
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(12) 
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(13) 



z L ez L 







R 2 

< 

E 

Pzz(z L 

)(i- 

“ £12 (- 


-x(z l ))+C l , 

(14) 


z L ez L 


where —2(1 — o) L < Cl < 2(1 —cr) L and tj(z L ), ei 2 (z L ) are 

computed via the distribution P z ^i.-i . A detailed derivation 
can be found in Appendix ICl 

By setting Cl = 0, we obtain what we call the L th 
order approximation of the outer bound and we denote it by 
C™ m (L). Clearly, C™ m (L) approaches C™ m exponentially fast 
in L. CT(L) is of finite-letter form and thus computable. 
The number of parameters in the corresponding feasibility 
problem is exponential in L. Since the approximate rate-region 
approaches the outer bound exponentially fast in L, it can give 
good approximations for reasonable values of L. 

Remark 4. For many examples, the parameter a in © turns 
out to be close to zero. In order to give meaningful bounds, L 
needs to grow large (in the order of L = 10000). Nevertheless, 
we observe in numerical examples that the variational distance 
decays much faster than ensured by ©. 

IV. Achievability 

We develop codes that achieve the outer bound in Sec. [Ill] 
The coding strategy is based on the coding techniques 
discussed in ED- More precisely, it uses network coding 
techniques of J71, [20] and a proactive coding strategy that 
was proposed in ED for the problem with causal channel 
state information. We discuss this coding scheme in our own 
framework to give an alternative viewpoint and as we believe 
the analysis is simpler than the one provided in l2ll . 

A. Queue Model 

To analyze the coding scheme, we build on the idea of 
tracking packets through a network of queues, as done in 
171 , |8l . The transmitter has two buffers, Q J 1 ' and Q^\ to 
store packets destined for Rxi, RX 2 , respectively. We consider 
dynamic arrivals, where packets for Rxi, RX 2 arrive in each 
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Fig. 2. Networked system of queues. 

slot according to a Bernoulli process with probability R\, f? 2 , 
respectively. These packets are called original packets. An 
analysis for more general arrival processes is possible. The 
transmitter maintains two additional buffers, and Q^\ 
for packets that were received by the wrong receiver only. 
Buffer Q'> 1 contains packets that are destined for Rxi and 
were received at RX 2 only, and vice versa for Q^\ If both Q^ 
and are nonempty, the transmitter can take a packet from 
both queues and send their XOR combination. Such a linear 
combination of original packets from and (f.f ] is called a 
coded packef]. Coded packets are useful to both receivers, for 
each can decode a desired original packet upon reception of 
the coded packet. Another necessary coding operation (see GO 
Example 2]) is as follows. The transmitter takes one packet 
from Qi"\ e.g. p^\ and one packet from Q^\ e.g. y/ 2 ), 
and sends the XOR combination of the two; i.e. y/ 1 ) + jZ 2 Z 
The packets involved have not been transmitted before; hence, 
we call this action proactive coding or poisoning l24l . A 
poisoned packet is not immediately useful for any receiver. 
However, it becomes useful together with a remedy packet 
that enables decoding of the original packet involved in the 
poisoned packet. For example, assume that the poisoned packet 
pi 1 ) -p pi 2 ) was received at RX 2 . If, at a later stage, the 
corresponding remedy is received at RX 2 , both yZ 1 ) and 
yZ 2 ) can be decoded at Rx 2 . Therefore, upon arrival of the 
poisoned packet at RX 2 , yZ 1 ) becomes as useful to RX 2 as 
yZ 2 Z Since yZ 1 ) is desired also at Rxi, it is more efficient to 
send pW rather than yZ 2 ^ in later channel uses; i.e., yZ 2 ) could, 
in principle, be replaced by the remedy packet yZ 1 ). 

Remark 5. If the poisoned packet is received only at Rx ; , 
then the remedy packet is pW, for j,j € {1, 2}, j j. If the 
poisoned packet is received at both receivers, the remedy is 
p(t) or pi 2 ). We fix it to yZ 1 ). 

Remedy packets are useful to both receivers. We put remedy 
packets into additional queues and Q^\ These two 

lr The original packets arriving to 1 may be coded as well in the sense 
that error-correcting codes have been employed by lower layers. We use 
the term coded packet to emphasize that information packets for different 
receivers have been mixed. Alternative terms could be network-coded or 
inter-session-coded packet. 


queues are conceptually the same queue to track the remedy 
packets. We draw them separately to account for correct packet 
arrival for each receiver separately. 

The system exit for Rxy is represented by buffer . Once 
a packet reaches Q± \ it leaves the system. These buffers are 
always empty by definition. 

The full networked queuing system is shown in Fig. |2] 
This queuing system is drawn as a graphical tool to track 
the packets that are sent and received. We restrict the set of 
actions at the transmitter to A = {1,2, 3,4, 5}: 

• A t = 1: send an original packet for Rxi from Q \ 1 

• A t = 2: send an original packet for Rx 2 from Q\ 

• A t = 3: send a coded packet from and 

• A t = 4: send a poisoned packet from Ql 1 ' and (/f 1 

• A t = 5: send a remedy packet from Qg or 

B. Packet Movement and Network Flow 

As the algorithm evolves in time, packets flow on the 
network of Fig. [2] Packet movements that are due to 
A t = 1,2,3 are based on Q, and are discussed in detail 
in 0. ED. & Packet movements that are due to A t = 4, 5 
are based on ED- We discuss them in our framework in the 
following. 

Packets may move to and (}'■?’ only if they are involved 
in a poisoned packet that is received by at least one of the 
receivers. To explain the packet movement from to Q^\ 
for example, consider two cases: 

( i ) Packet yZ 1 ) is a remedy packet (i.e., the poisoned packet 
was received at RX 2 ). In this case, yZ 1 > is moved from 
to Qg 1 -* (to be transmitted in later channel uses). 

(ii ) Packet yZ 1 '> is not a remedy packet (i.e. the poisoned 
packet was received only at Rx!). In this case, as 
discussed, yZ 2 ) is as useful as yZ 1 ) to Rxg. So yZ 1 ) may 
be replaced by yZ 2 ) and moved from Q ^ to Q^\ 

Remedy packets leave Qg when A t = 5, according to 
Table U This table appears in a similar form in |2l| . 

For clarification, we elaborate the case where the remedy 
packet sent with A t = 5 is received only at Rxi. A remedy 
packet can only be sent if a poisoned packet, say pi 1 ^ +p( 2 \ 
was previously received at Rxi or Rx 2 . The remedy packet is 
either pA) or p^ 2 \ We consider the two cases separately: 

1) If ' 1 is the remedy packet, then its arrival at Rxi (and 

not at RX 2 ) is captured by —> Q^\ Furthermore, 

since p ^ is a remedy packet, it is as useful to RX 2 as 
p( 2 Z So its arrival at Rxi lets us move the packet from 
Qg 2 ) to Q^\ because Rxi knows p^\ 

2) If /Z 2 - 1 is the remedy packet, then the poisoned packet 
pW+pi 2 ) must have been received only at Rxi. Arrival of 
p( 2 ) at Rxi (and not at RX 2 ) lets Rxi decode pW, hence 

Qg 1 '* —> . Also, since p^ is received at Rxi and not 

( 2 ) (2) 

at RX 2 , we have the packet movement Qg —> Q\ . 

Finally, we outline two algorithms that ensure network 
stability for rates in 
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(a remedy packet sent) 

received 
only at Rxi 

received 
only at Rx 2 

received 
at Rxi, Rx 2 

Packet movement 

q? ) ^q1 2) 


Qr^Qr 


TABLE I 

Packet movement for remedy packets in and . 


Theorem 3. Rate-pairs are achievable if 

(Pi + <5, P 2 + 6 ) G C™ for (5 > 0. 

C. Probabilistic Scheme 


The first scheme is a probabilistic scheme that bases 
its decisions on the past L feedback symbols. We prove 
achievability of C™ m (L) for any integer L > 0 which 
approaches C™ m exponentially fast in L. 

Fix L to be an integer and consider an encoding strategy 
that bases its decisions on the past L feedback symbols Z*T\. 
The decisions are random and independent from previous 
decisions, according to a stationary probability distribution 
^ > A t \z t ~ 1 L ^ a \- L )’ a G {1,..., 5}, z L £ Z L . To simplify 
notation here, we write P(a\z_ L ) for P At \ z t -1 (a| z L ) and 
P(z L ) for P z l(z l ). Based on the chosen probabilities, 
each link of the network in Fig. [2] has an effective 
capacity. This is the maximum number of packets that, on 
average, can go through each link. Denote the capacity 
of the link between Q y r and Q\ by c rl , where 
(r,l) G {(1,2), (1,3), (1,4), (2,4), (3, 2), (3,4)} and 
j £ {1,2}. According to the packet movement in Sec. IIV-BI 
we calculate the following link capacities for j £ { 1 , 2 }: 


C 12 = E P U i )( e ife i ) ~ ei 2 (z L ))P(j\z L ) (15) 

z l £Z l 

4 = E nz L ){l-e 12 {z L ))P{A\z L ) (16) 

z L ez L 

4 = E ^u L )(i-^fe L ))^ 0 'U L ) (i 7 ) 

z L GZ L 

4 = E ^(l L )(l-e,U i ))P(3U i ) (18) 

z L eZ L 

C 32 = E p U L )( e j{z L ) - ^2 (z l ))P(5\z l ) (19) 

z l GZL 

4 = E P{z L ){l-e 3 {z L ))P{^\z L ). (20) 

z L &Z L 

Using a max-flow analysis similar to |20| , the rate pair 
(Pi, R 2 ) is achievable if there exists a distribution P At \ z t-i 
such that the following flow optimization problem is feasible: 


Rj<f$ 
Aj) , Aj) fU) 

J 12 J32 — J 24 

f U) ^ f(j) 
J 13 — 32 

fU) <r U) 

J rl — rZ 


+/;?+ 

fti) 

J 14 

( 21 ) 



( 22 ) 

4 . fU) 

+ j 34 


(23) 

V(r, l)£< 

f(l, 2), (1,3), (1,4)1 
{(2,4), (3, 2), (3,4)/ 

j VjG{l, 2}. 



(24) 


This is a classical flow optimization problem where link 
capacities can be adjusted by P A ^ z t-i^. Note that the flow 


networks for Rxi and RX 2 are coupled only through P At \ z t-i ■ 
Using the max-flow min-cut duality theorem, the above flow 
problem is equivalent to the following min-cut problem: 


Rj < 4 
Rj < 4 
Rj — C V2 
Rj — c 14 


+ c U) 

+ c 13 

+ 4 

(25) 

+ c (j) 

-1- C 14 

_i_ jj) 

-r c 24 

(26) 

1 Jj) 

-1- C 14 

, Jj) , Jj) 

' c 32 ■ c 34 

(27) 

1 Jj) 

c 24 

. c 0) 

^ c 34 j 

Vi G {1,2}, (28) 


where {25} - {28} correspond to different cuts that separate 
Q[ :) from QE j £ {1,2}. 

The achievable rate-region is thus given by the feasibility 
problem defined in {25} - {28} over the set of probabilities 

P(l| 2 i ),...,P( 5 |z i ), z L G Z L . 

Proposition 1. Suppose P At \ z t -1 is feasible for a rate pair 
(Pi, P 2 ) G ^{^“(P). There exists a distribution P* t _i for 
which 

• (Pi, P 2 ) is feasible and 

• the bounds in {26} and {27} are redundant. 

The proof of Proposition |T| is given in Appendix 151 Hence, 
a rate pair (Pi,P 2 ) can be achieved if there is a distribution 
P. 1 ryt—i such that 


Rj < E^U L )( 1 -e i (E))[ J p (ilE)+ J P( 3 E)+ J P(5k 1 ')] 

z L ez L 

(29) 

Rj < E P fe L )( 1 -ei2U i ))[P(iU I ')+P(4|l 1 ')] • (30) 

z L ez L 


One can verify that the region described by {29} - {30} 
is equivalent to C™ m (L) by choosing a feasible set of 
distributions P(l|z L ),..., P(5\z L ), z L G Z L such that 

P(1\z l ) + P(3 \z l ) + P(5\z l ) = x(z L ), Vz L G Z L (31) 
P(2\z L )+P(3\z L )+P(5\z L ) = y{z L ), Jz L G Z L . (32) 

Remark 6. Note that the mapping from x{z L ), y{z L ), 
z_ L £ Z L to Pa,\z l defined in {37} - (132} is not unique. In 
particular, for some valid choices of P At \z L the constraints 
in {26} and (1 27} will not be redundant. Nevertheless, 
Proposition [I] tells us that one can always find a distribution 
P* 

A t \z L 

• that satisfies {37} - {32} and 

• for which {26} and (1 27} are redundant, 

for any values of x(z L ), y(z_ L ) z L £ Z L . For a given set of 
x(z L ), y{z L ) z_ L G Z L , we first choose a distribution P At \ Z L 
satisfying {37} - {32} and then determine P*from P At \z L - 

This shows that a feasible set of distributions 
P(l|l L ), • ■ •,P(5|l L ), £ Z L can be found if 

(Pi,P 2 ) G Cft m (i) and hence C” em (P) is achievable. 










At Weight depending on queue lengths and 7A 1 = z _ t 1 
~ [1 - - Q$) 

2 [1 - e 2 {z t - 1 )]Q^ t + e x2 (z t - 1 )(Q { i) ~ Q%) 

3 [1 - ei+ [1 - e 2 (z*- l )]Q% 

4 [1 _ e^fe*- 1 )] (q$ - Q$ + Q$ - Q$) 

5 e 12 (z t - 1 )(Q^} - Q$) + [1 - eife*- 1 )]^^ 

+fla(£ XQ§ -Qa 3 !) - [1 -22 )]Qi 2 t _ 

TABLE II 

Deterministic scheme with A t G A q[^ denotes the number of 

PACKETS IN QUEUE AT TIME t. 

D. Deterministic Scheme 

For the probabilistic scheme, one must compute the optimal 
set of probability distributions in advance. This set depends 
on Ri, R 2 which might be unknown to the transmitter ahead 
of time. Furthermore, in the probabilistic approach, it might 
happen that an action is chosen but there is no packet in 
the corresponding queues to be transmitted. We avoid both 
drawbacks by a max-weight backpressure-like algorithm (25), 
ll26l that bases its actions on both the queue state and the 
feedback. In each slot t, a weight function is computed for 
each action and the action with the higher weight is chosen. 
Table [II] lists the weights for each action depending on the 
current queue state Q_ t and the previous feedback state Z t_1 . 
The queue state Q_ t is a vector with one entry for each queue 
in the network containing the number of packets of each queue 
at time t. Note that the values of ei(V _1 ), e 2 (V -1 ) and 
ei 2 (z t_1 ) can be computed recursively. 

Proposition 2. The strategy in Table [TT| stabilizes all queues 
in the network for every (R\ + S, R 2 + 6) £ C™ m , S > 0. 

Hence, the strategy in Table [TT] provides an implicit way 
to compute the capacity region. The proof of Prop. [2] in 
Appendix [E] uses a finite-horizon Lyapunov drift analysis |26) 
that is adapted to account for previous observations (rather 
than the current channel state). 

V. Conclusion 

We studied the two-receiver broadcast packet erasure 
channel with feedback and hidden channel memory. The 
channel memory evolves according to a Markov chain that is 
not observable by the transmitter. We developed outer bounds 
that are achievable with two outlined coding schemes. 
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Appendix A 
Proof of TheoremQ] 

Theorem Q] is similar to the outer bound in |20| . We outline 
the proof here. The joint probability mass function of the 
system then factorizes as 

n 

PwiWzX 71 S n YpY : p Z™ Zg = Pw 1 Pw 2 Ps 0 J^[ Ps t |S t _i ' 

t=1 

‘ P X t \Z t 1 - 1 Z t 2 - lP Z 2 ,t\S t Pz 1 , t \Z 2 , t S t PY 1 , t \X t Z 1 , t PY 2 , t \X t Z 2 , t - 

The corresponding Bayesian network is shown in Fig. [T] We 
write Pz*- 1 ^ 1 ) as -P(P _1 ) and bound 

Ri-S < -I{Wy,Y?) 

n 

< -UWx-YPZ™- 1 ) 
n 

n 

= - E 1(W 1 - 1 Y 1:t Z 2 , t -i\Y*- 1 Z!f 2 ) 

1 t =1 

= lj2[ I ( W Y Z 2,t-l\ Y t 1Zt l~ 1Z 2~ 2 ) 

1 t= 1 

+I(W 1 ;Y 1 , t \Yt 1 Zt 1 )] 

n 

= J2 -imY^YT'zt'zt 1 ) 

t— 1 n 

1 " 

)/(Wi; Y 1>t \Y*-\ Z*- 1 = z t ~ 1 ). (33) 

1 t= 1 £*-1 

In the above chain of inequalities, (a) follows because Z f ^ 1 
is a function of Yj 4-1 and W\ — Y*~ l Z\~ [ Z!^ 2 — Z 2 ,t -i forms 
a Markov chain for every t = 1 ,n. Markov chains can be 
checked in Fig. [2 Similarly, we write 

Ri~S< —I(Wy,Y™Y£'\W 2 ) 

n 

1 n 

= - E HWyYiMw^y*- 1 ) 

11 t =1 

1 n 

- -EE P^WYY^tlW^Yt 1 ,^- 1 ^- 1 ). 

n t=lz_t-i 

where (a) follows because _Z t_1 is a function of Y*~ [ , K 2 t_1 . 
We next obtain a relationship using a derivation similar to f20l 
Lemma 1]. For every t = 1,..., n, z t_1 £ Z t_1 , j = 1, 2, we 
have 


= (1 - e j (~ * _1 ))I(Wi;X t \ Yj 4-1 , Z 4-1 = 2 t_1 ), 

I(Wi; Y ut Y 2 >t | W 2 Y *- 1 Yt 1 , 2 ^ = 2*" 1 ) 

= (1 - ei 2 (^ 1 ))I(W 1 ;X t \W 2 Y{- 1 Y£- 1 ,Z t - 1 =z t ~ 1 ). 

Define the variables: 

u U) fetr-1 ) = i(w i; x t | Yt 1 , Z*- 1 = z ‘- 1 ) 

V U) ( 2 *-1) = j(Wi; X t \ W 2 Y*- 1 Y^- 1 1 Z t - 1 = z t ~ 1 ) 


( 34 ) 





Appendix C 

Approximation of Outer Bounds 



Fig. 3. Bayesian network for the two-receiver broadcast packet erasure 
channel with memory and ACK/NACK feedback, for n = 3 and d = 1. 
Dependencies due to feedback are drawn with dashed lines. 


From BOl Lemma 2], for every t = 1 = 1,2, 

z t_1 E we have 

+ n ( ^U t_1 ) < 1- (35) 

We thus obtain the following feasibility problem as an outer 
bound on the achievable rate-region: 

0 <uW (z*- 1 ), v (j) (z*- 1 ) <1 Vt, i~ x (36) 

u (j) (z*- 1 ) + v® U*" 1 ) <1 V t, z*" 1 (37) 

1 " 

Rj <-EE p fe H )( 1 -^ i " 1 )) ,,fa1 (^ 1 ) (38 > 

n t=l z *- 1 

1 " 

1 t= 1 jt-l 

The inequality (l37l ) can be made tight without changing 
the rate region. By setting ti( 1 i(z i ) = x(z_ L ) and 
u,( 2 \z L ) = y(z_ L ), the above characterization can be written 
as in <0 - ©. 


Appendix B 

Derivation of CorollaryQ] 


The corollary follows from Theorem U and the following 
derivation: 


E r )- p (zt 14- 


'!) 


=E 

=E 




E ( p N* t_1 ) - p ( s tl^:i)) p U t |s t ) 


< E l p (^ t_1 ) - | E p (^) 


= El p (^k t " 1 )- p (^kt=i)| 


(40) 


We derive only the first inequality for the outer 
approximation of the outer bound in (ITTb . The other 
inequalities follow similarly. We write P z t-i( 2 t_1 ) as P(z t_1 ) 
and bound 


o ( < p vv f p fe t ” 1 )( 1 -eiU t - 1 )) 

1 -n ^ 4-f \ xliW^XtlYf- 1 ,#- 1 = P- 1 ) 


t= 1 **-i 
n 


( P ( p *) ((! ~ eiUt-i) + 2(3 - a ) L ) 

- n 2^2^\ xiiw^XtlY*- 1 ,#- 1 = z*- 1 ) 


t=1 z*- 1 


<-E 

n 1 ' 


t=l 


V P fe i - 1 )(l-ei(^-L)) 

2^ x/(Wi; X t II?" 1 , Z 4 - 1 = z t ~ 1 ) 

+ 2(1 - a^IiW^XtlYt 1 ,#- 1 ) 


K sr' ( p z$A L (-t-i) e i(lt-z,)) 




where (a) follows from (l33l >. The sequence 2" is stationary, 
i.e. P z t-i (z L ) does not depend on t but only on z_ L . Hence 
one can simplify the above equations using a time-sharing 
argument with time-sharing random variable T: 


*.<£! 


P^fe L )(l-ei(l 1 ')) 

, x/(Wi;X T |Ty L T - 1 Z T - c '- 1 


7 t-i 
’ —T—L 


+ 2(1 — a) L 


(41) 


Define 

x(z L ) = I(W 1 -,Xt\TY?- 1 Z t - l -\zI-\ =z l ). 

We obtain the outer approximation of the outer bound in (ITTb : 

Pi <2(l-a) L + '^2PzL{z L )(l-e 1 (z L ))x(z L ). (42) 

Z L 

An inner approximation can similarly be derived with a 
negative sign for 2(1 — a) L . The outer bound is sandwiched 
between the inner and outer approximation. 


Appendix D 

Proof of Proposition!]] 

We again write P(a\z L ) for P& t \z t_1 ( a k L ) an d define the 
variables Aj, Bj, Cj, Dj, j € {1, 2} as the cut values defined 
in (l25b - (l2lb and shown in Fig. 0] 


Aj — C^2 + Ci3 + 

= E p fe i )( 1 - ^ L )) [ p uk L )+m~ L )] 

z L £Z L 

d _„ I Aj) I Aj) 

Bj — C13 + C 14 + C 24 

= Y. P k L )[^-^2(z L ))P{A\z L ) 

z L £Z L 



















= E P &) ((! - ^{z L ))P{A\z L ) - (1 - ej(z L ))P(5 \z l )) 

z l £Z l 

= E P 4) ((! - ^2(z l ))P(5\z l ) - (1 - ej fe L ))P(5U L )) 

z l GZ l 

>0, (47) 

where the last inequality is due to 1 — £ 12 ( 2 L ) > 1 — tj{z L ) 
for all G Z L and j = {1, 2}. ■ 

Remark 7. Note that one cannot simply set P*(4|z L ) = 
P*( 5|z L ), Vz L G Z L , to ensure (l45l i. By the mapping in 
(OH - ( OH . this would imply x(z L ) + y{z L ) > 1 , \/z_ L G Z L 
and hence permit only a limited set of values for x(z L ) and 
y(z L ). This additional constraint leads to a smaller rate region, 
as derived in | 20 |. 


Fig. 4. Queue network and cuts. 


+ (l-e j (z L )) [P(j\z L ) + P(3\z L )\ 

r - r W 4- 4- 4- 

— c 12 ■+" c 14 c 32 “r c 34 

= E PU L )( 1 - ei 2 U L )) [P(jk L ) + P(5|* L )] 

z l £Z l 

Dj = 4 + 4 + 4 

= E PU L )(l-e i fe I '))[P(jU i )+P(3|£ I ')+P(5U 1 ')] 

z L &Z L 

We omit the superscript (}) for C 13 to emphasize that 

c 13 — c 13 — c 13- 

Our goal is to show that for any link capacities cty 
and associated cut values Aj, Bj, Cj, Dj induced by a 
distribution P, there is another distribution P* 

with associated link capacities c'B* and cut values A*, P*. 
Cj, D* such that 

min {A*, P*, O'*, P*} = min {Aj, Bj, Cj, Dj} (43) 

min { A* , B* , C* , D* } = min { A* , P* } . (44) 

That is, the minimal cut value does not change under 
P* ,„t-i, but the cuts P* and C* are redundant. The next 

A t\A t -L J J 

lemma states one sufficient condition for redundancy: 


Lemma 2. The cut values Bj and Cj are redundant for both 

j = 1,2 if P At \ z t-i satisfies 

C 13 = 4 + 4 V j G {1, 2}. (45) 

Proof: The condition in ( l45l > is equivalent to 

E P{z L ){l-e 12 {z L ))[P{±\z L )-P(f\z L )\=Q. (46) 

z L eZ L 

One can verify that Cj = Aj in this case, so Cj can be omitted. 
Additionally, Dj < Bj because 

Bj ~ Bj 

• + 4 + 4 - (4 + 4 + 4) 


For any given distribution P At \ z t-i, we choose P* 
such that G 

P*(l|« i ) = P{l\z L ) (48) 

P*(2\z l ) = P(2\z L ) (49) 

P*( 4 U L ) = P(4|4) (50) 

P*(3|4) + P*(5\z L ) = P(3\z L ) + P(5 \z l ) (51) 


By choosing P*( 3 | 2 L ) ^ P(3|z L ), P*( 5 | 2 L ) 7 ^ P(5|z L ), the 
link capacities 4^*> 4** and 4^*> 7 ^ {1,2} are varied. The 
other link capacities are not affected, hence 


-12 


= c 


U) 

12 ! 


„ 0 > 

-13 


= C 


O') 

13 5 


4 * = 4, j e {1,2}. (52) 


„ 0 ) 


In the following we therefore omit the superscript * for those 
capacities that stay constant with P A , 7 t-i and P* t _ x . 

Note that A* = Aj, D* = Dj, j = (1, 2} because 

• A} is not affected by changing P*(3\z L ) or P*(5\z L ) 
and 

• D j depends only on the sum P*(3\z_ L ) + P*{5\z L ) that 
is kept constant in (l5lt . hence 


So) 1 So) _ So)* , So)* 

C 24 ~r C 34 — C 24 -I- C 34 . 


(53) 


By changing P*(3\z L ) and P*(5|z L ) under the 
sum-constraint <E0» one can obtain the maximal and 
minimal link capacities for 4 ** and 4 **> as follows: 

!) 4 * = 0 , 4 * = D o - 4 > 7 = {f^ 2 }’ by setting 
P*(3|l L ) = P(3|i L ) + P(5|i L ) and P*( 5 | 2 L ) = 0, 
Sz L ez L . 

2 ) 4 * = 0 . 4* = Bj - 4 , j = { 1 , 2 }, by setting 
P*(3 |z l ) = 0 and P*(5|4) = P(3|2 L ) + P(5|i L ), 
Vz L G 

The link capacities are linear with respect to P* . t _ x , hence 

A t\A t _L 

any convex combination of ^Dj — cfl , 0^ and ^0, Dj — cf} ''j 

can be obtained for ^c 4 *i c 34 *) ■ 

We will now show that choosing P* ,„ t -i as in (l48l) - (l5ll) 

At \SZ +- r. _ 

suffices to ensure the desired criteria (l43b - ®. We have to 
distinguish two cases: 




Case I: Aj < Dj for at least one j £ {1,2}. 

This happens if we have 

c[j + Cl 3 < C 24 + c 34 for some 3 C {1, 2}. (54) 

In this case, we choose P*(5\z L ) such that c 3 j* = C 13 — cj]*. 
By Lemma0 this is sufficient to ensure the criteria ( 1431 - (|44| >. 
We can always find such values for P*(5\z_ L ) because, by 
definition of Case I in (l54t . there is a j £ {1, 2} for which 

Dj - eg > eg + ci 3 . (55) 


The LHS of (155} is the maximal possible link capacity for 
c 34 *■ The RHS of ([55} is larger than (or equal to) C 13 — eg*, 
which is the desired value for eg*. We can adjust 
between 0 and Dj — cjj/. As C 13 — c- 


'34 


u 32 


lies in this interval, we 
can choose values for P*(5\z L ) such that eg* = C 13 — eg*. 

Case II: Dj < Aj for both j = {1,2}. 

In this case the sufficient criterion in Lemma [2] cannot be 
guaranteed, since it is possible that there are no values for 


P*(5\z_ l ) such that C13 = eg* + c[ 


AA* 

'34 


To satisfy the criteria (143} - ([44}. we need for both j = {1, 2}: 


Dj < B *, hence cj}* < C 13 (56) 

Dj < C*, hence eg* < c g* + eg (57) 

In this case we choose P*(5\z L ) such that max jcg*, eg*! 
is as large as possible, but at most equal to C 13 , in order not 
to violate {56}. Two sub-cases need to be distinguished: 

Case Ha: One can choose P* (b\z L ) such that 

f ( 1 )* ( 2 )*) 

max | C 34 ,cy j = C 13 . 

The condition in (156} is satisfied by construction, so we 
have to check only ([57} . Note that the following inequalities 
always hold for Case Ha: 


eg + c 34 — c i 2 + c i 3 , because Dj < Aj , and 

eg* + eg** < eg + C 13 , because of (|53} . Hence, 

eg* < eg + (c 13 - eg*) . (58) 

For condition (157} to hold, the term ^ci 3 — should be 

smaller than (or equal to) 4 a * for both j = { 1 , 2 }. 

For j max = argmax je{12 } egg we have C 13 - cg max) * = 0, 
satisfying the condition. 

For j min = argmin je{12 } egg we have c 13 - cg min) * > 0. 
We next show that C 13 — c g mm )* < cg nin ^* holds for this case 
as well. As C 13 = 4} max g we have 


r , _ C?min)* _ (jmax)* _ (jmin)* 

<-13 <-34 — <-34 <-34 

= E F (Z L ) (^>nU L ) - ej—U L )) p *(5\z L )>0. 

z L ez L 


The following statement shows that C 13 — c ^ min )* < 

(jmin)* ( jmin )* 

<-13 <-34 <-32 

= E P U L )[D2U L )^^ x (z L )\P*(5\z L )<0 (59) 

z l £Z l 


as ei 2 (z L ) - e 7max (z L ) < 0 for all z L £ Z L , j £ 
{1,2}. This shows that the choice of P*(5|z i ) such that 
max jcgg 4 ?*} = c i 3 i s sufficient to achieve the criteria 
( 143} - ( 144} in this case. 

Case lib: Choosing the maximal P*(5\z L ) = P(3\z L ) + 
P{5\z L ) Mz L £ Z L leads to max jcgg cg*j < c i 3 - 
This immediately satisfies the condition in ([56}. As P*(5|z i ) 
is maximal, we have P*(3\z L ) = 0 £ Z L . Hence, 

cg^* = 0, V) £ {1,2}, satisfying the condition ([57} . This 
completes the proof. 


Appendix E 

Proof of Proposition [2] 

(j) 

With slight abuse of notation, let Q \ t denote the number of 
packets stored in buffer Qg at time t. Obviously, Q\j £ No- 
Define 

Q.t = (QS,Qg,Qg,Qg,Qg,Qg) e < ( 60 ) 

Because Qg = Qg = 0 by definition, the vector Q_ t 

determines the queue state at time t. 

(i) 

Let F' r i t denote the number of packets that can travel from 
buffer Qj' 1 to Q[ J> in time slot t. Clearly, we have Fjj\ £ 
{0,1}. F)p t is a deterministic function of the action At and 
on the random erasure events Z f . Because of the erasures, 
the values of f are unknown to the transmitter before the 
transmission. The actual number of packets travelling on the 

(j) (a\ ~(i) 

link from buffer QA to Q\ ' in time slot t is denoted P r / t . 

pjll may be different from Fjjj, e.g. because buffer Qj 1 is 

’ ~(i) • . 

empty and no packet can be transmitted. F' r :{ t is a deterministic 
function of A t , Z_ t and Q_ t . We have pg = 1 {Q.g > 0}P r g t , 
hence gg < F%) t . 

(A\ ~( 

To model external packets arrivals, let F 311 = f’ 01 t denote 
the indicator random variable if a packet arrived in queue (j: l! 
during time slot t. f 0 [ t is independent of all other random 
variables in the system, and we have 

HF^ t ]=Rj. (61) 

The flow divergence [27} Chapter 1.1.2] at buffer Qj' 1 is 
defined as 


A ( ? = E F /£ t -E^S- (62 > 

m k 

The flow divergence is thus the difference of the number of 

(j) 

packets that can depart from buffer Q) minus the number of 
packets that can arrive at Q) . 

(j) 

The number of packets in queue Q\ evolve according to 


Qg + i = ^?-Egg+Eg 


?U) 

kl.t 


< 


og-E^? 




(63) 




Ol 0-2 Q 3 Qa 



We bound the individual terms inside the inner expectation 
of ( l68l > by using ll28l Lemma 4.3], which states that for any 
nonnegative numbers v,u,n,a satisfying v < [u — fj] + + a, 
we have 

v 2 <u 2 + ++a 2 ~ 2 u(n — a). (69) 

We apply this lemma and combine it with ) 2 = Ff T 

(j) 

because F+ __ is either 1 or 0 and obtain: 


Fig. 5. Bayesian network of the queuing system. Actions at time t are 
permitted to depend on the previous feedback messages Zf ~ 1 and the current 
buffer state Q t . Note that Q t denotes the buffer state before action At is 
executed. 


where [x] + = max(x, 0). There is an inequality rather than 
an equality because some queue Qf might be empty, hence 

pW) < +A 

r kl,t ~~ r kl,f 

Each coding scheme that we will define leads to a different 
evolution of the queue state Q_ t , as the actions at time / might 
be different. Our main interest is whether all queues in the 
network are stable. A queue Q t is strongly stable if |26l 

1 " 

limsup — V E[Q t ] < oo. (64) 

n —>oo ri 

A network of queues is strongly stable if all queues inside the 
network are strongly stable. 

The action at time t can depend only on the current queue 
state Q_ t and on the previous channel feedback messages Z t_1 . 
Hence it can be represented by a distribution -P.A t |£ 2 t z ‘- 1 - All 
dependencies are depicted in the Bayesian network in Fig. 0 
Define the Lyapunov function L(<2 t ) as 

2 3 

M.Qt) = EE(y ( J) 2 (65) 

3=1 1=1 

and the '/'-slot conditional Lyapunov drift A(Q_ t ) as 

A(Qt) = E [UO-t+T) - HQt) \Qt] ■ (66) 

The expectation is with respect to the possibly random actions 
A t , A t+1 , ..., A t+T -i, erasures Z t ,Z t+1 ,... ,Z_ t+T _x and 
queues £2 t+ i, < 2 t+ 2 > ■ • • >Q.t+T■ A( Q t ) measures the expected 
reduction or increase of the queue lengths from slot t to slot 
t + T, for some fixed constant T, conditioned on Q t , and is 
useful to prove strong stability. 

We split the 7'-slot conditional Lyapunov drift A (Q_ t ) into 
the telescoping sum 


t+T-l 


A(<2 1 )= E E[m T+1 )-L(Q T )\Q t 


T=t 

t+T-l 


= y E E[L(fi r+ 1 )-L(fi r )|fi T Z' 


T—1"| 


Q_ t 


(67) 


( 68 ) 


where the last equality is because Q_ T+1 is a deterministic 
function of (J_ r - A T ,Z_ T and Q_ T A T Z_ T — Q_ t ZJL_l ~ Q-t. forms 
a Markov chain for r > t. 


L( Qr+i) - HQ-r) < 12 - 2 y y QgDg ( 70 ) 

3=1 1=1 

where Df is the flow divergence defined in (1621 ). 

We use (l68l ) together with (1701) to obtain 


t+T-l 


2 3 



A(o_ t ) < y E 

E 

i 2 - 2 yy^g 

QrZ^ 1 

Q-t 

T=t 


3=11=1 




(71) 


The terms in the inner expectation are 


\ n U)\ n 7 r+_(Fi[At=j\Q_ T Z_ T \, (7T-1\\ 

[ D i,tUtZ \-\+Vv[A t =A\Q T ZJ+f L_ei2 ^ >)- 


Rj 
(72) 


Df T \Q T Z T - 1 \=PT[A t = 3\Q T Z T - 1 ](l - +Z T +) 

_(Rl[A t =j\Q_ T ZJ X ] V ( 7 T- In ( 7 t- 1n\ 

V4-Pr[A t = 5|a T ^ T "V ( '^ ) ^ 

(73) 


Df T \a T z T ~ { 


fPr[A t =5\Q T Z^] V 
\-Pi[A t =A\Q_ T Z T - 1 })" £l2( - 


(74) 


where the expectation is with respect to a distribution 
Pa t |<2 z r ~ 1 • All relations follow because Z_ T — ZJ~ l — A r Q r 
forms a Markov chain. 

The strategy in Table [TT| finds the tightest upper bound in 
( 1711 ) under the assumption that actions can depend on Q_ T and 
ZJ- 1 . Hence, any stationary probabilistic scheme that bases its 
decisions only on Z+T_ h , according to a distribution P A | Z r-i , 
leads to a looser upper bound on A(<2 t ). 

We introduce notation for the average flow divergence: 



dfi+r-L) 


(75) 


where the expectation on the LHS is with respect to a 
distribution P A z t-i . This expression does not depend on 
the queue state Q_ T and ZJ~ L ~ 1 . 

The relations in d72l ) - (f74l ) are adapted to 


r ]U)/yr—\ \ /Pr[A( j\Z_ T _i\ A /1 (7 T ~ 1 A 

d ! ~ {+PiiA t =A\ZlZ 1 L ]) [ 

df{Z+} L ) = Pr[A t =3|^y](l - +Z+\)) 


- R.; 


(Pr[A t =j\ZJ~\] 

V+Pr[A t = 5|iy L 


i(Z+- 


T— 1 

L 


)-ei2 {Z+A)) 























(76) 


t+T-1 


a m< E 12 - E 


(a) 


T=t 

t+T-l 


2 3 


2 EEQuME^E) 

j=i ;=i 


Q-t 


< E 12 + 12(r — f) — E 


2 3 


2 EE^4 J) (r-L) 

i=i l =i 


Q-t. 


< 12T + 6T 2 

= 12T + 6 T 2 


(d) 

< 12T + 6T 


2 


( e ) 

< 12T + 6T 


2 


2 t+T-l 3 

EE E Pr [£T-l=£--l\ a t ]E^^feEJ 

j=l r=t /- 1 Z=1 

2 3 t+T-l 

2 EE < 5?iE^ W fe i ) E Pr [^r-L =l L |a t ] 

J = 1 Z = 1 2+ T=t 

2 r EE +4 

i=ii=i 

2 3 

2t(5- £l , t );e^qE 

j=i ;=i 



(77) 


(78) 

(79) 

(80) 




r—1 
L 


) 


/Pr[+ = 5|^E L ] 
V-Pr[+=4|^I 1 L 



£ 12 ( 2 ;^+)). 


Hence we can continue with the chain of inequalities in 
( 1761 - < 1 SOb on the top of the page. 

For step (a) we have used the property that buffer level 
can decrease by at most one per slot. The expression inside the 
expectation in (1771 ) does not depend on Q_ T anymore. Step ( b) 
writes out the expectation and rearranges terms. Because the 
sequence Z_ n is stationary and also the probabilistic strategy 
is stationary, cf'f 1 (zZZ. l L ) does not depend on r but only on 
the realization Z+ L = z L . This is used in step (c). 

Define the mixture distribution 

, i+T-l 

M T {z L \g)=- E P zj-_\ (81 > 

T=t 


The constant T can be chosen large enough such that 


E 2 x d\ j \z L ) (+++) - M T (z L \g)) 

and g_. This follows because 


is small, for all 


E d\ j \z L )(Pz L (z L )-M T (z L \g ;.)) 

Z L 

<E | d PV)| | Pzl{z L ) - M t {z l W)\ 

Z L 

<E \ P z l (z L ) - m t(z l \sl)\ < e l ,t, Va (82) 


of the corresponding Markov chain, for any initial distribution 
of Pu t \Q_ t - If (A- is distributed according to the steady state 
distribution, so is its component ZP_ L , i.e. according to 
P z -r- 1 (z L ). This distribution does not depend on r because 

Z n is stationary, so we can replace it with Pz^(z L ). If 
converges to P z 1 ,, so does the Cesaro mean in (l82l) . 
In general the constant T must be significantly larger than L. 
For step (d), we replace the expression 

E^ d\ J \z L ) Ettl" 1 Pr [ZJ-\ = z L I Q-t] by its lower 
bound T (E,!. 4 j) (z L )PzL{z L )) - T £l , t . 

For step (e), note that if the rate pair is in the interior 
of the L* order approximation of the outer bound, i.e. if 
(f?i + 5, i ?2 + 8) € C+ m (L), then there exists a constant 
S > 0 that goes to zero when 5 —> 0 such that 

E P ^ i te L M 0) fe i ) > S, VZ €{1,2,3}, j € {1,2}, 


where <5 should be chosen such that 5 > £l,t- 

Using the result in (l80l > and the law of total expectation, we 
can bound 


E [L(Q t+T ) - L(Q*)] =E[A(Q t )|a 

2 3 

< 18T 2 — 2T(S — £l,t) EE e 


Q$ 


(83) 


2=1(=i 


for some el,t > 0. Hence, the variational distance between 
P z l and Mt needs to become small. 

Recall that the channel state Markov chain is irreducible 
and aperiodic. Define the random variable Ut = (Sl_ L Z%Z L ). 
Note that Ut is Markovian and the corresponding Markov 
chain is irreducible and aperiodic. One may verify that U T +1 
is independent of Q_ t given U T for r > t and hence the 
distribution of U T converges to the steady state distribution 


Summing over all time slots t = 1 ,,n yields 

n T 

E E [L (Q t+T ) - L(&)] =E E [ E CQ t+ J - HQt)] 


t =1 

n 2 3 


<18nT 2 — 2T(5 — £l,t) EEE e [E: 

t= 1 2=1 1=1 


( 84 ) 



















Rearranging terms gives 


1 

n 


n 2 3 


£££«w“] 


< 


9 T 


8 - e L : 


EL E [ L (g t )] 

2{8-e L , T )Tn' 


and taking a lim sup with respect to n on both sides 
proves strong stability of the queuing network, given that 
l/^E«=i E[L(£2 t )] < oo. This is true for any finite constant 
T and E[L(^)] < oo. 
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